Business Analyst Community & Resources | Modern Analyst

What does “good” look like for an AI feature—and how do you test it?

Adrian M. — Mon, 29 Dec 2025 21:01:00 GMT

“Good” for an AI feature is not just accuracy. “Good” means the feature delivers the intended outcome reliably, safely, and consistently within defined boundaries—and you can prove it.

What “good” looks like (the BA checklist)

A well-defined AI feature typically has targets in these areas:

1) Outcome (value)

What business outcome improves (time saved, errors reduced, conversion increased)?
What metric proves it?

2) Quality

Accuracy (or usefulness) at an agreed threshold
Consistency (similar inputs → similar outputs)
Latency (response time)
Cost (especially for AI inference)

3) Safety & compliance

No prohibited content
No leakage of sensitive data (PII, credentials)
No unsafe actions (especially if the AI can act in systems)

4) Trust & transparency

Users can understand why it produced an output (explanations, sources, confidence)
The system handles uncertainty gracefully (“I’m not sure, here’s what to do next”)

5) Operability

Monitoring in place (drift, anomalies, failure rates)
Rollback/kill switch criteria defined

How you test it (practitioner approach)

Testing AI features blends traditional testing with evaluation:

A) Test with representative scenarios

Build a test set of real-world cases (including edge cases)
Include “hard cases” and “red team” attempts (misuse, injection, sensitive data prompts)

B) Define measurable acceptance thresholds
Examples:

“At least 90% of outputs meet rubric score ≥ 4/5”
“High-risk actions require human approval 100% of the time”
“No PII appears in outputs in the test set”
“Confidence below X triggers escalation”

C) Test behavior under uncertainty

What does it do when data is missing?
Does it hallucinate or does it ask for clarification?
Does it choose safe defaults?

D) Monitor after release
Because AI behavior can change, “good” must be maintained:

track drift,
review exceptions,
adjust guardrails and evaluation sets.

What distinguishes a functional requirement from a solution or design constraint?

Adrian M. — Sat, 10 May 2025 01:02:00 GMT

Functional requirements describe what the system must do, while solution or design constraints set the boundaries on how the system may be built.

Definitions

Functional requirement (FR): An externally visible behavior that satisfies a business need. Example: “The system shall calculate the total order price, applying state-specific sales tax and any active promotions.
Solution / design constraint (SC): A rule or limitation that restricts technology, architecture, process, or environment. Example: “The calculation must reuse the existing SAP tax engine and complete in ≤ 2 seconds.”

Key Differences

Aspect	Functional Requirement	Solution / Design Constraint
Purpose	Deliver a capability or service	Limit the implementation space
Focus	Outcomes, interactions, user value	Tools, standards, timing, budget, compliance
Validation	User acceptance tests, use-case walkthroughs	Design reviews, architectural conformance, audits
Flexibility	Multiple ways to meet the need	Narrows or eliminates options

Why It Matters

Clarity: Separating FRs from constraints prevents “design by requirement,” keeping the what independent of the how.
Innovation: Teams can explore creative solutions as long as they respect immovable guardrails.
Traceability & Testing: FRs map to user acceptance tests; constraints map to design or non-functional verification, reducing gaps at audit or release time.

Illustrative Scenario

A retailer wants faster checkout.

FR: “The POS system shall apply loyalty discounts and display the final total to the cashier.”
SCs:
1. “The solution must run on existing Android POS tablets.”
2. “Data transmission must be encrypted to meet PCI-DSS requirements.”
3. “Discount calculation response time shall not exceed 1.5 seconds.”

The FR articulates the desired capability, leaving room for various algorithms or UI designs; the SCs fence those choices with technology, regulatory, and performance limits.

What is Scope Creep and how do you manage it?

Chris Adams — Mon, 10 Oct 2022 17:30:00 GMT

Scope creep occurs when additional features are slipped into a project after the project scope is defined and the project has started. The intent is generally good with the hope of making the delivered product better, but these changes can lead to the project running over time and/or over budget. How you manage scope creep depends on the type of project.

What are some pros and cons of requirements reuse?

Chris Adams — Mon, 13 Dec 2021 17:28:00 GMT

Requirements reuse is when the same written requirement is used repeatedly within a single project, across a product line, or even across an enterprise. Requirement reuse can be as simple as an analyst copy/pasting requirements from a previous project, or it can be as sophisticated as having a central repository of requirements that documents link to.

What are the Business Analysis Knowledge Areas?

Chris Adams — Tue, 07 Dec 2021 20:56:00 GMT

This question is an easy way for a hiring manager or an HR professional to screen out candidates who may have under-invested in their career or may lack understanding of the business analysis profession. The 6 Knowledge Areas are:

Elicitation & Collaboration
Requirements Analysis & Design Definition
Solution Evaluation
Requirements Life Cycle Management
Business Analysis Planning & Monitoring
Strategy Analysis

What is an FRD (Functional Requirements Document)?

Adrian M. — Fri, 16 Apr 2021 15:50:00 GMT

What you ever wondered, or have been asked, “What is an FRD?”

At the core, an FRD or Functional Requirements Document serves as a contract for formal statement, between the business stakeholders and the technology team, on an application’s functional requirements. The FRD is generally produced by the technical team in response to the business requirements (captured in a BRD - Business Requirements Document, PRD - Product Requirements document, or some other suitable format).

The key purpose of an FRD is to bridge the gap between business and technology. It’s where project stakeholders and the technical development team meet. The creation of the FRD forces and ensures collaboration and address both sides of the coin:

Business - it restates the business requirements in terms of functional features and capabilities to be supported by the new system or platform. This ensures the project team understands the business requirements and are on their way to implement a solution which addresses the business needs or problems.

Technology - it captures key technical constraints and commitments as well key interfaces to external systems

While created by the solution team, the FRD should be solution independent (in general) aka it should express what the application should do and not how it should do it. The FRD should not commit the technical team to a specific design.

The Functional Requirements Document (FRD) is one way to express functional specifications and define the requirements and functional solution direction of software solution. The FRD is not the only way - there are other functional specification formats and templates, depending on methodology and organizational needs::

Systems Requirements Document (SRS)
Use Cases
User Stories
High-Level Impact Assessment

While FRD templates vary, some of the more common elements of a functional requirements document are (just a suggestion):

Business process and workflows - provides a view of the desired target state business process models outlining which steps in the process will stay as-is and which steps will be either automated or impacted by the solution at hand.

Functional Requirements - these are the traditional “the system shall…” type requirements and can be broken into multiple subsections such as:
- User Requirements
- Regulatory Requirements
- Compliance Requirements
- User Interface Requirements

Operational Requirements - while some may be functional in nature, the operational requirements tend to focus more on administrator level capabilities and needs:
- Role Based Security
- Audit Trail
- Configuration

Data Requirements - describe the data which needs to be supported by the system. This section may include logical data models, data migration and conversion requirements. It is not uncommon to also see data flow diagrams describing the conceptual flow of data.

Non-Functional Requirements - these generally describe attributes of the system which, when done right, may never be observed by the users which use the functional capabilities.
- Security Requirements
- Performance Requirements
- Capacity
- Fault Tolerance
- Data Retention
- Recoverability
- Additional sections or information which may be found in an FRD:
- Background
- Scope
- List of Project Stakeholders
- Assumptions
- Constraints

Your turn!

Do you use Functional Requirements Documents?
How do you structure your FRD?
What is an FRD, for you?

Is it a Defect or is it an Enhancement? How to manage the disagreements.

Chris Adams — Tue, 06 Apr 2021 19:25:00 GMT

“It is a defect, or an enhancement” is one of those arguments in software development that never seems to satisfy anyone when someone makes a decision. I’ve gone so far as to say it is a defectment. It’s really a combination of both. However, that doesn’t necessarily answer the question - nor provide guidance to a resolution.

What are SMART requirements?

Adrian M. — Fri, 14 Aug 2020 01:57:00 GMT

At first glance, smart requirements are requirements which are well thought out, well written, selected with the end user in mind, consider the customer's experience, consider trade-offs when it comes to implementation (such as cost, time).

However smart requirements, or better put SMART Requirements refer to an acronym whose goal is to help you come up with really good requirement statements:

Specific
Measurable
Attainable (appropriate, actionable, achievable)
Realistic
Testable and Traceable

There are other mnemonic acronyms which help a business analyst ensure the requirements are the best that they can be:

CLEAR
- Collaborative
- Limited
- Emotional
- Appreciable
- Refinable

PURE
- Positively stated
- Understood
- Relevant
- Ethical

CPQQRT
- Context
- Purpose
- Quantity
- Quality
- Resources
- Timing

Share a time when you had to change the mind of the business stakeholder. Why did you have to do it? How did you do it?

Chris Adams — Sun, 12 Jul 2020 19:57:00 GMT

As you respond to a question like this, keep the following center of mind:

Business outcomes must be met - explain how value is still being provided
Use visuals - process flows, mockups, both is best
Stick to the messaging - don't let someone rattle you

What approach do you use to evaluate the impact of a requirement change on the project?

Chris Adams — Sun, 10 May 2020 22:05:00 GMT

Business Analysts need to show that they approach any issue in a structured and methodical manner. This includes evaluating the impact of changing requirements on a project. When a requirement change occurs, business analysts often assist the project manager in the evaluation of it’s impact. Together they determine the appropriate course of action. Proposed changes should be evaluated based on how they might impact eight distinct project areas:

What strategies can the BA and project team use to facilitate stakeholder sign-off on requirements?

Chris Adams — Mon, 14 Jan 2019 15:05:00 GMT

There are several steps that the BA and project team can take beyond just publishing requirements with a sign-off deadline, in order to ease the process for stakeholders and expedite approvals:

Make sure the right people are signing-off on requirements. It sounds simple and obvious, but this can be one of the biggest contributors to challenges in obtaining requirements sign-off, particularly on complex multi-stakeholder projects.
Help stakeholders understand the sign-off process. Provide a clear explanation of the commitment that stakeholders are making to the project by signing off on requirements. Be sure to communicate the process for identifying and managing subsequent changes after sign-off (which may be just a reminder of the project’s established change control procedures).
Help stakeholders interpret the requirements documentation. Where feasible and applicable, provide supporting or reference material to assist interpretation and understanding of the requirements. This may include glossaries and data dictionaries, diagrams, etc. It is also very helpful to maintain a log of all decisions that are made throughout requirements definition, and publish this decision log with the requirements to avoid revisiting past decisions.
Make sure stakeholders have enough time to review the requirements during sign-off. Plan ahead for review and sign-off cycles as much as possible. Remind stakeholders of this commitment 2-3 weeks prior to each review cycle, so they have enough advance notice to plan for the time needed.
Foster collaborative ownership on multi-stakeholder projects. If a single document contains requirements that are shared or jointly owned, individual stakeholders may be reluctant to sign-off on requirements that they do not individually own. Joint review sessions can be used for collaborative discussion with the goal of obtaining verbal approval from the group during the session. Stakeholder questions or concerns can be addressed through the group review, which will help expedite sign-off from each individual participant.

--
Sandy Lambert
Business Architect
LinkedIn Profile

In which document do you include the Class Diagram (business requirements, functional requirements, software specification document)?

sonavi — Mon, 12 Nov 2018 16:20:00 GMT

Just like any other diagram, the Class Diagram is just a tool at the disposal of the analyst. In the absence of a set process, it is at the analyst’s discretion to determine when to use a class diagram. Therefore, in which analysis artifact/document a class diagram should be included depends on its use.

How does the Business Analyst role change on an Agile project compared to projects using other software development methodologies?

Chris Adams — Mon, 12 Mar 2018 15:02:00 GMT

The role of the BA should actually change very little between different software development methodologies, although the tools and techniques used by the BA can vary according to the needs and attributes of any given project or development lifecycle.

The core responsibilities of a BA on a software development project include requirements elicitation, requirements analysis and requirements management – regardless of the project methodology. The type and format of requirements documentation are just tools, and a good BA has a wide range of tools at his or her disposal.

Once a system is developed is it reasonable to document changes with simple updates to screen mockups?

Chris Adams — Sun, 10 Sep 2017 22:16:00 GMT

This question implies that the benefit of foregoing the creation of a more complete requirements specification document is a significant amount of time savings. But what might we be losing in the process.

Screen mockups alone don’t clearly document requirements. Instead, they reflect a decision made by the system designer to satisfy a particular requirement. Often when someone views the mockup or updated system they may think the requirement is obvious when, actually, they have misinterpreted the true requirement.

What is RML (Requirements Modeling Language)?

Chris Adams — Sun, 10 May 2015 23:02:00 GMT

Requirements Modeling language is a collection of diagrams used to model software from the business analysis or product management perspective.

How do you ensure that you’ve identified all of the necessary project stakeholders?

Chris Adams — Sun, 08 Mar 2015 22:12:00 GMT

Who is requesting the project deliverable?

Who is the champion of the project?

Who controls the budget for the project?

Who is currently responsible for the business process that may be impacted by the project?

Who is currently responsible for the systems that may be impacted or replaced by the project?

Whose job is directly impacted by the delivery of the project?

When implementing a purchased solution should your requirements and design documents cover features that were not requested but come with the package?

Chris Adams — Sun, 07 Sep 2014 18:22:00 GMT

When implementing a purchased solution at a minimum each additional feature needs to be assessed to determine its potential positive or negative impact.

Why is requirements traceability important on a project?

Chris Adams — Fri, 06 Jun 2014 04:07:00 GMT

Tracking and managing requirements traceability offers several valuable benefits to a project:

Minimize Risk and Manage Costs. Requirements traceability lets you identify and analyze the upstream and downstream impacts of any requirements change, so those impacts can be effectively managed and mitigated.
Manage Scope. Traceability provides an understanding the relationships between requirements and identifies stakeholders who have a vested interest in each requirement, which supports better decision-making about project scope and proposed changes throughout the project.
Improve Quality. Traceability allows individual requirements to be managed within the context of the full solution, and helps preserve the integrity of the end-state solution throughout each development cycle or product release. It also helps ensure that all requirements are fully tested and enables effective integration, system and regression testing.
Increase Stakeholder Engagement. Requirements are clearly linked to vested stakeholders, giving greater visibility of the value achieved for each stakeholder group throughout the project lifecycle.
Reduce Development Time. The ability to identify and manage groups of related requirements results in more efficient and effective planning, and streamlines product development.

--
Sandy Lambert
Business Architect
LinkedIn Profile

How is the Requirements Management and Communication knowledge area of the BABOK v2.0 defined?

Chris Adams — Sat, 06 Mar 2010 21:34:00 GMT

How do you define the term Requirement?

Chris Adams — Sat, 06 Feb 2010 20:55:00 GMT

What is the 4 +1 View Model as it relates to system modeling?

Chris Adams — Sat, 09 May 2009 21:59:00 GMT

The 4 + 1 View Model is a predefined set of views for organizing the design and architecture of a system. It was developed in 1995 by Philippe Kruchten, formerly the Director of Process Development at Rational Software.

The 4 + 1 View Model gets its name from the 4 primary views and 1 supporting view that are used to capture and communicate different aspects of the system.

The 4 primary views are:

Logical View: this view describes the functionality of the system in terms of its static structure and dynamic behavior.
Development View: this view describes the system from a programmer’s perspective and is concerned with the organization of physical code, its main modules, and their dependencies.
Process View: this view focuses on the runtime behavior of the system and the elements of the system that relate to process performance. It includes aspects important to scalability, throughput, and process response times to name a few.
Physical View: this view shows the system from a system engineer's point-of-view. It is concerned with the deployment of software components across the physical architecture including computers and devices , as well as communication between these components.

The 1 supporting view is:

Use Case View: this view describes the functionality of the system from the perspective of external actors.

Why is requirements traceability important?

Chris Adams — Fri, 08 Aug 2008 15:36:00 GMT

Requirements traceability is the ability to follow and audit the life of a requirement, in both a forward and backward direction; from its origins, through its realization in the design and functional specifications, to its eventual development and deployment and use, through subsequent rounds of modification and refinement.

If a requirement is changed or modified, using a requirements traceability matrix the analyst can identify those areas within the functional specifications that are impacted by the requirement and make the appropriate modifications. In addition, the analyst can determine if the portion of the specification being modified traces back to any other requirements. This is important to ensure that the new modification doesn’t break any existing requirements within the system.

Similarly, sections of the specification can be traced to areas of the physical code to ensure that all required changes are made during the development process. Test cases can also be traced back to functional specifications. When functional specifications are changed test cases nearly always need to be updated to ensure complete and accurate testing can take place.

--
Chris Adams
LinkedIn Profile

Tell me about a time when you were faced with a requirement/enhancement that was not feasible. How did you handle the situation?

Adube — Wed, 06 Aug 2008 18:13:00 GMT

Even if you know that a requirement is not feasible, be sure not to simply dismiss it away. Your responsibility as an analyst is to maintain a strong working relationship with the business representatives. Approach this situation in a way that makes it clear that the decision isn’t a personal one, but that it’s based on a well informed position with associated costs.

Clarify the requirement. Remember, often when a “requirement” is elicited it is really stated in the form of a solution. So take the time to verify the true requirement.

Identify why the requirement isn’t feasible. If the new requirement and the previous requirements cannot be accommodated at the same time regardless of the technical implementation, then present this information to the business. Do your best to explain the technical limitations using non-technical language that the business can understand.

Most often however, the implementation of a new requirement isn’t feasible due to the prohibitive cost that would be involved in making the technical changes required to support it. Estimate the associated cost and present this information to the business. This is helpful for a few reasons. First, you may have assumed that the cost outweighed the need for the requirement, but the business may feel differently depending on the business goal driving the requirement. Second, if the business agrees the cost of implementing the new requirement is just too prohibitive, then you as the analyst have made your case while making the business feel empowered and involved in the decision making process.

Are use cases the functional requirements or do you think functional requirements are different from use cases?

everest — Fri, 02 May 2008 03:29:00 GMT

It is generally accepted that use cases, specified in narrative form (also known as use case specifications), depict functional requirements. This is because a use case, via the main and alternate flows, shows how a user interacts with a system in order to achieve a desired result.

That's exactly the purpose of a "functional requirement" to describe the functions and behaviors that a system is or should be capable of.

Therefore, if use cases are used and narrated in detail for a project, there is no need for separate documentation to describe the functional requirements because the totality of all the use cases represent the set of functional requirements for a given system/project.

What are some of the benefits of a requirements management tool?

Emy — Fri, 29 Feb 2008 13:13:00 GMT

A requirements management tool supports the analyst in documenting project requirements, versioning requirements, managing the requirement change process, tracing requirements through the analysis and design process, and tracking the status of requirements.

Some examples of requirement management tools are:

Enterprise Architect,
Requisite Pro,
Telelogic,
Caliber-RM

And here is a more extensive list of top benefitsof a requirements management tool:

Management and tracking of requirements including traceability, versioning, auditing, history

Requirements workflow and approval

Colaborative requirements development, even when team members are remote

Automatic generation of requirement documents and specifications

Generation and traceability of test cases and test scenarios

Tools with graphical design options enable requirements to be represented as diagram and/or user interface mockups and prototypes

End-to-end impact analysis and coverage validation

Why is it important to prioritize requirements?

Analyst Kris — Wed, 20 Feb 2008 10:33:00 GMT

The main reason why it is important to prioritize requirements is because like most of us, organizations have finite resources (money, time, employees, etc.)

Since not all requirements can be implemented right way, the business needs to determine which requirements should be addressed first. Without going into the details on how to perform requirements prioritization, we should point out that the ultimate determining factor is ROI (return on investment).

How much money am I going to make/save if I implement this requirement. Yes - most business are about money but this works even for non-financial driven requirements (such as how many more people am I going to convince to vote for candidate X if I implement requirement # 1 vs. requirement # 2).

In order to determine the ROI the business analyst needs to look at a number of factors (aka attributes of each requirement) including but not limited to:

how important (biggest savings or gain) is the requirement to the business that is, if the cost to implement each requirements was the same, what would be the order of execution,

how much would it cost to implement each requirement,

how much would the business lose if they don't implement a given requirement (for example if the requirement is as a result of a law how much could it cost the company if it were to get into litigation for now complying with the law?),

how long would it take to implement a given requirement; note that longer duration does not necessarily mean increased cost, however longer duration could mean a lost business opportunity which, in turn, might call for not implementing the requirement in the first place.

etc.

Additional resources:

First Things First: Prioritizing Requirements

Requirements Prioritization Introduction

What does a requirements document contain?

sonavi — Fri, 08 Feb 2008 01:17:00 GMT

Short Answer: Requirements! ;-)

Seriously... The reality is that there isn't such as thing as a standard requirements document template to help guide the business analyst in the creation of this document.

The format of a Requirements Document vary depending on the type and size of project, type of organization, maturity of the business analysis team, use of specialized requirements management tools, type of methodology and development process(agile vs. RUP vs. structured analysis), etc.

Having said that, here are some common types of information found in many requirements documents:

Background/History
Scope and Objectives
Regulatory Requirements
Business Level Requirements
- Strategic
- Tactical (Interoperability)
- Operational (Process related mostly)
Stakeholder and User Analysis
User Requirements (the abilities that the users need)
Functional Requirements
Non-functional Level User Requirements
Assumptions/Constraints
Risks and Dependencies
Solution Options
Business Glossary (the nouns and noun-verb phrases of the business)
Reference to Business Rules
Reference to Business Case/Vision
Use Case Models

One more observation: requirements documents are also known by a variety of names which, at times, mean the same thing and, other times, refer to totally different documents:

Requirements Document
Business Requirements Document (BRD)
Software Requirements Document
Software Requirements Specification (SRS)